Accounting for the Uncertainty in the Evaluation of Percentile Ranks

In a recent paper entitled “Inconsistencies of Recently Proposed Citation Impact Indicators and how to Avoid Them,” Schreiber (2012) proposed (i) a method to assess tied ranks consistently and (ii) fractional attribution to percentile ranks in the case of relatively small samples (e.g., for n < 100). For example, a journal may contain a limited number of (e.g., ten) reviews in a year. In that case, the highest possible percentile in this category in this year would be based on (9/10) or 90%. Rousseau (2012) proposes to define this highest possible rank as 100% by including the ranked paper in the ranking, and thus to consider (10/10) as the highest possible rank. Leydesdorff & Bornmann (in press) argue that this definition of a percentile is not compatible with standard definitions (Hyndman & Fan, 1996) and too easily leads to an upward drift in percentile rankings so that non-cited papers could be greatly overestimated in a citation analysis (Zhou et al., in preparation).